BIOST 561: unit testing

Lecture 5

Announcements

Why unit testing?

Why do we need a formal way to run our tests?

Your responses to the Q4 in HW3

Suppose your teammate gave you a function to find the maximal clique in an adjacency matrix (i.e., the set of nodes that forms the largest clique). You are not told the typical size and characteristics of these adjacency matrices beforehand. Your job is to make sure this function is correct since you and your teammate are about to give this function to your manager, who will then give it to another division in your company to use. Your performance review will depend highly on whether or not other people in your company can reliably use your function.

In a short paragraph, write down ways to ensure your teammate’s function is “correct.” Please list at least four different ways you can test this function. You can interpret this notion of “correct” very liberally – this question is purposely framed to be open-ended.

Types of unit tests

Checking that the function outputs something that is the correct type

Simple checks to make sure result is within the correct range

Making sure the function runs on many different inputs

Making sure the function gets the correct answer for carefully crafted problem

Making sure the function gets the correct answer with the help of visualization

Making sure the function gets the correct answer with many randomly generated problems

Stress testing to make sure the function handles corner cases gracefully

Testing the behavior when there is deliberately no unique answer

Comparing against another known implementation

Comparing against another one of your implementations (possibly much more computationally intensive, but more transparent and definitely correct)

Math: Exploiting some mathematical property of your problem

Testing the timing of your function

Testing to make sure it errors when expected

Testing the default values

Testing the coding environment

Checking the documentation/literal coding

Checking the intermediary functions!

So… what does this show?

What’s the goal?

A personal note

A personal note

Setting up unit tests in R

library(testthat)
library(UW561S2024Example)

test_check("UW561S2024Example")

See this in:

An example of unit test

Let’s look into the unit tests in test_compute_probabilities.R: https://github.com/linnykos/561_s2024_example/blob/main/tests/testthat/test_compute_probabilities.R

context("Testing compute_probabilities")

# Unit test for compute_log_probabilities
test_that("compute_probabilities outputs correctly", {
 set.seed(10)
 # Mock data and parameters for testing
 data <- matrix(rnorm(20), nrow = 10, ncol = 2) # 10 samples, 2-dimensional
 means <- matrix(c(0, 0, 5, 5), nrow = 2, byrow = TRUE) # 2 components
 variances <- c(1, 2)
 proportions <- c(0.5, 0.5)

 probabilities <- compute_probabilities(data, means, variances, proportions)

 # Test if probabilities sum to 1 for each sample
 expect_true(all(abs(rowSums(probabilities) - 1) < 1e-6))

 # Test if probabilities are within the valid range [0,1]
 expect_true(all(probabilities >= 0 & probabilities <= 1))

 # Test for handling of a single sample (edge case)
 single_sample <- data[1, , drop = FALSE] # Prevent dropping to lower dimension
 probabilities_single <- compute_probabilities(single_sample,
                                               means,
                                               variances,
                                               proportions)
 expect_true(dim(probabilities_single)[1] == 1)
 expect_true(all(abs(rowSums(probabilities_single) - 1) < 1e-6))
})
context("Testing compute_probabilities")

# Unit test for compute_log_probabilities
test_that("compute_probabilities outputs correctly", {
 set.seed(10)
 # Mock data and parameters for testing
 data <- matrix(rnorm(20), nrow = 10, ncol = 2) # 10 samples, 2-dimensional
 means <- matrix(c(0, 0, 5, 5), nrow = 2, byrow = TRUE) # 2 components
 variances <- c(1, 2)
 proportions <- c(0.5, 0.5)

 probabilities <- compute_probabilities(data, means, variances, proportions)

 # Test if probabilities sum to 1 for each sample
 expect_true(all(abs(rowSums(probabilities) - 1) < 1e-6))

 # Test if probabilities are within the valid range [0,1]
 expect_true(all(probabilities >= 0 & probabilities <= 1))

 # Test for handling of a single sample (edge case)
 single_sample <- data[1, , drop = FALSE] # Prevent dropping to lower dimension
 probabilities_single <- compute_probabilities(single_sample,
                                               means,
                                               variances,
                                               proportions)
 expect_true(dim(probabilities_single)[1] == 1)
 expect_true(all(abs(rowSums(probabilities_single) - 1) < 1e-6))
})
context("Testing compute_probabilities")

# Unit test for compute_log_probabilities
test_that("compute_probabilities outputs correctly", {
 set.seed(10)
 # Mock data and parameters for testing
 data <- matrix(rnorm(20), nrow = 10, ncol = 2) # 10 samples, 2-dimensional
 means <- matrix(c(0, 0, 5, 5), nrow = 2, byrow = TRUE) # 2 components
 variances <- c(1, 2)
 proportions <- c(0.5, 0.5)

 probabilities <- compute_probabilities(data, means, variances, proportions)

 # Test if probabilities sum to 1 for each sample
 expect_true(all(abs(rowSums(probabilities) - 1) < 1e-6))

 # Test if probabilities are within the valid range [0,1]
 expect_true(all(probabilities >= 0 & probabilities <= 1))

 # Test for handling of a single sample (edge case)
 single_sample <- data[1, , drop = FALSE] # Prevent dropping to lower dimension
 probabilities_single <- compute_probabilities(single_sample,
                                               means,
                                               variances,
                                               proportions)
 expect_true(dim(probabilities_single)[1] == 1)
 expect_true(all(abs(rowSums(probabilities_single) - 1) < 1e-6))
})
context("Testing compute_probabilities")

# Unit test for compute_log_probabilities
test_that("compute_probabilities outputs correctly", {
 set.seed(10)
 # Mock data and parameters for testing
 data <- matrix(rnorm(20), nrow = 10, ncol = 2) # 10 samples, 2-dimensional
 means <- matrix(c(0, 0, 5, 5), nrow = 2, byrow = TRUE) # 2 components
 variances <- c(1, 2)
 proportions <- c(0.5, 0.5)

 probabilities <- compute_probabilities(data, means, variances, proportions)

 # Test if probabilities sum to 1 for each sample
 expect_true(all(abs(rowSums(probabilities) - 1) < 1e-6))

 # Test if probabilities are within the valid range [0,1]
 expect_true(all(probabilities >= 0 & probabilities <= 1))

 # Test for handling of a single sample (edge case)
 single_sample <- data[1, , drop = FALSE] # Prevent dropping to lower dimension
 probabilities_single <- compute_probabilities(single_sample,
                                               means,
                                               variances,
                                               proportions)
 expect_true(dim(probabilities_single)[1] == 1)
 expect_true(all(abs(rowSums(probabilities_single) - 1) < 1e-6))
})
context("Testing compute_probabilities")

# Unit test for compute_log_probabilities
test_that("compute_probabilities outputs correctly", {
 set.seed(10)
 # Mock data and parameters for testing
 data <- matrix(rnorm(20), nrow = 10, ncol = 2) # 10 samples, 2-dimensional
 means <- matrix(c(0, 0, 5, 5), nrow = 2, byrow = TRUE) # 2 components
 variances <- c(1, 2)
 proportions <- c(0.5, 0.5)

 probabilities <- compute_probabilities(data, means, variances, proportions)

 # Test if probabilities sum to 1 for each sample
 expect_true(all(abs(rowSums(probabilities) - 1) < 1e-6))

 # Test if probabilities are within the valid range [0,1]
 expect_true(all(probabilities >= 0 & probabilities <= 1))

 # Test for handling of a single sample (edge case)
 single_sample <- data[1, , drop = FALSE] # Prevent dropping to lower dimension
 probabilities_single <- compute_probabilities(single_sample,
                                               means,
                                               variances,
                                               proportions)
 expect_true(dim(probabilities_single)[1] == 1)
 expect_true(all(abs(rowSums(probabilities_single) - 1) < 1e-6))
})

Using devtools::check()

Three more things: refactoring your code

Second: “contracts” (checking the inputs and outputs)

cleanup_na_matrix <- function(mat){
 stopifnot(is.matrix(mat), all(is.numeric(mat)))
 
 n <- nrow(mat)
 p <- ncol(mat)
 mat <- sapply(1:p, function(j){
   .cleanup_vector(mat[,j])
 })
 
 return(mat)
}

Alternative:

cleanup_na_matrix <- function(mat){
 if(!is.matrix(mat) | !is.numeric(mat))
   stop("mat is not a numeric matrix")
 
 n <- nrow(mat)
 p <- ncol(mat)
 mat <- sapply(1:p, function(j){
   .cleanup_vector(mat[,j])
 })
 
 return(mat)
}

Third: the limitations of unit testing

Words of wisdom

In-class exercise: Bootstrap confidence intervals

bootstrap_lm_coef <- function(X, y, alpha = 0.05, nboot = 1000){
  # something is done here
}

List of types of unit-tests

Testing for completeness:

Testing for correctness:

Elements #8 through #11 could all benefit from randomly generated inputs since you don’t really need to know what the exact output is to write a useful test!

Learn from other coders!

Additional resources

With the remaining time…